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What We Studied 


From 2004/05 to 2014/15, the number of English learner students in bilingual or English as a second language (ESL) 
programs in Texas increased by 47 percent. In the Houston Independent School District (SD)—the largest district in 
Texas and the seventh largest in the United States—30 percent of students are English learner students, and 92 percent 
of these students have Spanish as their home language. The number of students served by the district in bilingual or 
ESL programs grew at more than twice the rate of total enrollment during the course of the 2014/15 academic year. 
The number of first-year immigrants has more than doubled since 2011/12, and these newcomers lag behind other 
English learner students in terms of their English proficiency (Houston ISD, 2016). As a result of these factors, the 
district has been facing a critical shortage of bilingual-certified teachers. 


Aware of these challenges, the members of Regional Educational Laboratory (REL) Southwest’s English Learners 
Research Alliance sought to obtain clear and actionable information that districts can use when recruiting teachers and 
assigning them to schools and classrooms that serve large numbers of English learner students. To respond to the needs 
of the Alliance, REL Southwest designed this study, which uses data obtained from the Texas Education Agency 
(TEA) and Houston ISD. The study assesses whether a teacher’s certification type—being a certified bilingual teacher 
or a certified ESL teacher—and route to certification—alternative, post baccalaureate, traditional, or additional exam— 
correlate with academic growth and growth in English proficiency among English learner students whose home 
language is Spanish. It assesses 10 cohorts of grade 4 and 9 cohorts of grade 5 of Houston ISD English learner students 
between 2005/06 and 2014/15. 


How We Analyzed the Data 


The primary data source used for the analyses is the database maintained by the Texas Education Research Center 
(Texas ERC) at the University of Texas at Austin. This data contains statewide, longitudinal, student-level data. Also, 
data from TEA and Houston ISD were added and accessed through the Texas ERC. REL Southwest obtained access to 
student test scores, student demographics (i.e., race, ethnicity, age, gender, English learner student status, and poverty 
status [whether the student was eligible for the federal free or reduced-price lunch program]) teacher certification and 
demographic information, and school characteristics. 


The average grade 4 cohort in the analytic sample contained about 3,800 English learner students in 2006/07—2010/11 
and about 4,800 students in 2011/12—2014/15. The average grade 5 cohort contained about 2,700 English learner 
students in 2006/07—2010/11 and about 3,700 students in 2011/12—2014/15. In this study, grade 3 test scores served as 
a pretest for the grade 4 cohorts, and grade 4 test scores as a pretest for the grade 5 cohorts. All test scores were 
standardized to have a mean of zero and a standard deviation of one within each grade-year combination. Therefore, all 
figures illustrating assessment scores in the report are in standard deviation units. Student achievement models were 


TEXAS Education 


Page | 1 The University of Texas at Austin 
College of Education 


then constructed to estimate the association between teacher certification characteristics and English learner students’ 
growth in math, reading, and English proficiency. Student achievement models, sometimes referred to as value-added 
specifications, were formulated to predict student achievement growth and included teacher certification characteristics 
as predictors. These models calculated coefficients on certification type and route variables; included student, teacher, 
and school demographic covariates; and were estimated by grade (4, 5), subject (math, reading, English proficiency), 
and state assessment program (TAKS, STAAR, TELPAS). 


Research Questions: 

1. What are the value-added contributions to English learner students’ growth in math, reading, and English 
proficiency of bilingual- and ESL-certified teachers? 

2. What are the value-added contributions to English learner students’ growth in math, reading, and English 
proficiency of bilingual- and ESL-certified teachers trained through different routes to certification? 


The study team then examined (by grade, subject, and state assessment program) whether specific teacher certification 
types (research question 1) and routes (research question 2) were associated with larger achievement gains. To obtain 
an accurate estimate of the contribution of a teacher’s certification route on English learner students’ growth, the 
analyses needed to account for both certification route and certification type. The total certification (certification type 
and certification route) contributions to English learner students’ growth are reported in figures 2—6. Bilingual through 
alternative route and ESL through additional exam indicators were not included in the model and thereby served as the 
reference categories, the coefficients on the other type by route indicators measured the effects of the other routes 
relative to the reference route. 


What We Discovered 


Research Question 1: What are the value-added contributions to English learner students’ growth in math, 
reading, and English proficiency of bilingual- and ESL-certified teachers? 

In grade 4, certified bilingual teachers were associated with greater growth in math and reading than teachers with no 
bilingual or ESL certification (see table 1 and figure 1). The effect size is 0.12, and is equivalent to 21 percent of the 
average annual gain in math from grade 4 to grade 5 in nationally normed tests (Hill, Bloom, Black, & Lipsey, 2008). 
An additional reference to put this effect size in context is the size of the math achievement gap between grade 4 
English learner and non-English learner students in the 2015 National Assessment of Educational Progress (NAEP) 
Trial Urban District Assessment (TUDA) for Houston ISD: 12.3 standard deviations (U.S. Department of Education, 
2015). The effect size in reading is 0.04. To put this 0.04 effect size in context, the size of the reading achievement gap 
between grade 4 English learner and non-English learner students in HISD in the 2015 NAEP assessment was 15 
standard deviations, so the effect size of a bilingual teacher in grade 4 is less than | percent of the gap in reading 
between English learner and non-English learner students in the district. 


Table 1. Certified bilingual teachers were associated with English learner students’ growth in math/reading grade 4. 


Bilingual certification ESL certification 
Effect Effect 
Dyicexerd(eya) Effect size Direction Effect size 

Math it 0.12*** na -0.01 
Reading + 0.04* na -0.03 
English 

Grade 4 proficiency na -0.01 na 0.00 
Math = -0.08** na 0.02 
Reading na -0.02 na 0.01 
English 

Grade 5 proficiency na 0.04 na 0.00 


ESL is English as a second language. + and — stand for significant positive and negative effect, respectively. 
* Significant at p < .05, ** Significant at p <.01, *** Significant at p <.001. na is not applicable since the parameter is not significant. 
Note: The reference group is teachers who are not certified as either bilingual or ESL teachers. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 
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In grade 5, certified bilingual teachers were associated with lower math growth compared to teachers who were not 
certified bilingual or ESL, with an effect size of —0.08 (figure 1). In the case of reading, there is no evidence that 
certified bilingual teachers contributed more or less to grade 5 English learner students’ growth compared to teachers 
who were not certified bilingual or ESL. The estimates suggest that being assigned a certified bilingual reading teacher 
did not contribute significantly more or less to English learner students’ English proficiency compared to teachers who 
are not certified bilingual or ESL in either grade 4 or grade 5. Being assigned a certified ESL teacher did not contribute 
significantly more or less to grade 4 or grade 5 English learner students’ math or reading achievement growth 
compared to being assigned a teacher who was not certified bilingual or ESL (the reference group). Likewise, ESL- 
certified teachers did not contribute significantly more or less to English learner students’ English proficiency 
compared to teachers who were not certified bilingual or ESL in either grades 4 or 5. 


Figure 1: Certified bilingual teachers were associated with English learner students’ growth in math and reading in 
grade 4 on the State of Texas Assessments of Academic Readiness 
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ESL is English as a second language. STAAR is State of Texas Assessments of Academic Readiness. 
*Significant at p < .05. ** Significant at p < .O1. *** Significant at p < .0O1. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


Research Question 2: What are the value-added contributions to English learner students’ growth in math, 
reading, and English proficiency of bilingual- and ESL-certified teachers trained through different routes to 
certification? 

In grade 4, estimates suggest that bilingual teachers certified through alternative routes were associated with the most 
growth in math achievement compared to bilingual and ESL teachers certified through other routes and compared to 
teachers with no bilingual or ESL certification. 


Specifically, only two of the four routes to bilingual certification were associated with significant effects on math 
achievement growth: alternative routes (effect size = 0.17) and traditional routes (effect size = 0.12) (see figure 2 and 
table 2). Relating the magnitude of these effect sizes to the NAEP grade 4 math achievement gap between English 
learner and non—English learner students mentioned previously, the contribution of bilingual teachers certified through 
alternative routes to English learner students’ math growth is equivalent to 1.4 percent of that achievement gap. 


The contribution of bilingual teachers from traditional routes is 1 percent of that gap. Students assigned to bilingual 
teachers from the additional exam and postbaccalaureate routes did not experience significantly different growth in 
math compared to students assigned to teachers with no bilingual or ESL certification. 
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Figure 2. Bilingual teachers from alternative certification routes were associated with English learner students’ 
greatest math achievement growth in grade 4 
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ESL is English as a second language. STAAR is State of Texas Assessments of Academic Readiness. 
* Significant at p < .05. ** Significant at p < .01. *** Significant at p < .001. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


In grade 5, estimates suggest that bilingual teachers certified through the additional exam route and ESL teachers 
certified through traditional routes were associated with the least growth in math achievement compared to bilingual 
and ESL teachers certified through other routes and compared to teachers with no bilingual or ESL certification (see 
figure 3). Specifically, only two of the four routes to bilingual certification and only one of the four routes to ESL 
certification were associated with significant effects on math achievement growth, and in all three cases, the effect was 
negative: bilingual traditional (effect size = —0.15), bilingual additional exam (effect size = -0.19), and ESL traditional 
(effect size = -0.19) routes. Students assigned to bilingual teachers from alternative or postbaccalaureate routes did not 
experience significantly different growth in math compared to students assigned to teachers with no bilingual or ESL 
certification. Similarly, students assigned to ESL teachers from alternative, additional exam, or postbaccalaureate 
routes did not experience significantly different growth in math compared to students assigned to teachers with no 
bilingual or ESL certification. The analysis was not able to detect an association between bilingual or ESL teachers 
certified through other routes and math achievement growth in grade 5. In grade 5, estimates suggest that bilingual 
teachers certified through the additional exam route and ESL teachers certified through traditional routes were 
associated with the least growth in math achievement compared to bilingual and ESL teachers certified through other 
routes and compared to teachers with no bilingual or ESL certification (see figure 3). Specifically, only two of the four 
routes to bilingual certification and only one of the four routes to ESL certification were associated with significant 
effects on math achievement growth, and in all three cases, the effect was negative: bilingual traditional (effect size = — 
0.15), bilingual additional exam (effect size = —0.19), and ESL traditional (effect size = -0.19) routes. Students 
assigned to bilingual teachers from alternative or postbaccalaureate routes did not experience significantly different 
growth in math compared to students assigned to teachers with no bilingual or ESL certification. Similarly, students 
assigned to ESL teachers from alternative, additional exam, or postbaccalaureate routes did not experience 
significantly different growth in math compared to students assigned to teachers with no bilingual or ESL certification. 
The analysis was not able to detect an association between bilingual or ESL teachers certified through other routes and 
math achievement growth in grade 5. 


To determine the difference in achievement growth between students assigned to bilingual teachers from traditional 
routes and those assigned to bilingual teachers from the additional exam route, the effect size from the additional exam 
route teachers is subtracted from the effect size from traditional route teachers:—0.15 ——0.19 = +0.04. This difference 
in effect size would translate to the mean math score of students taught by bilingual teachers from traditional routes 
being at the 52nd percentile of students taught by a bilingual teacher from the additional exam route. 
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Figure 3. Bilingual teachers from the additional exam certification route and ESL 
teachers from traditional certification routes were associated with English learner 
students’ least math achievement growth in grade 5. 
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ESL is English as a second language. STAAR is State of Texas Assessments of Academic Readiness. 
* Significant at p < .05. ** Significant at p < .01. *** Significant at p < .001. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


As mentioned previously, having an ESL teacher certified through traditional routes also was associated with the least 
math achievement growth in grade 5 compared to ESL teachers certified through other routes and compared to teachers 
with no ESL or bilingual certification. The estimated effect size was —0.19 (figure 3 and table 2), which translates to an 
improvement index of —8. To put this effect size in terms of percentiles, it indicates that the mean math score of the 
students taught by ESL teachers from traditional routes would be at the 42nd percentile of the students taught by an 
ESL teacher certified through other routes or by a teacher with no ESL or bilingual certification. The analysis was not 
able to detect an association between ESL teachers certified through alternative, additional exam, and 
postbaccalaureate routes and math achievement growth in grade 5. In grade 4, the analyses suggest that bilingual 
teachers certified through traditional routes were associated with the most growth in reading compared to bilingual and 
ESL teachers certified through other routes and compared to teachers with no bilingual or ESL certification. 
Specifically, only one of the four routes to bilingual certification was associated with significant effects on reading 
achievement growth: traditional routes (see figure 4 and table 2). The effect size of bilingual teachers from traditional 
routes on reading growth is 0.08, which results in an improvement index of +3 and indicates that the mean reading 
score of the students taught by a bilingual teacher from traditional routes would be at the 53rd percentile of students 
taught by a bilingual teacher from any other route or by an ESL teacher from any route (figure 4). Results also 
indicated that bilingual teachers from alternative, additional exam, or postbaccalaureate routes and ESL teachers from 
any route were not associated with greater or lesser reading achievement growth compared with teachers with no 
bilingual or ESL certification. 


Figure 4. Bilingual teachers from traditional certification routes were associated 
with English learner students’ greatest reading achievement growth in grade 4 
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ESL is English as a second language. STAAR is State of Texas Assessments of Academic Readiness 
* Significant at p < .05. ** Significant at p < .01. *** Significant at p < .OO1. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


The results suggest that the different certification routes chosen by bilingual and ESL teachers could not explain any 
difference in their contributions to English learner student achievement in reading. The analyses on English proficiency 
outcomes in grade 4 suggest that bilingual teachers certified through postbaccalaureate routes were associated with the 
most growth in English proficiency compared to bilingual and ESL teachers from other routes and compared to 
teachers with no bilingual or ESL certification (see figure 5). Specifically, only one of the four routes to bilingual 
certification was associated with a positive and significant effect on English proficiency growth: postbaccalaureate. 
Two routes to bilingual certification were associated with negative and significant effects: traditional and additional 
exam (figure 5 and table 2). Students assigned to bilingual teachers from alternative routes did not experience 
significantly different growth in English proficiency compared to students assigned to teachers with no bilingual or 
ESL certification. Although ESL teachers from alternative and traditional routes also were associated with positive 
growth in English proficiency, their effect size was smaller than that of bilingual teachers from postbaccalaureate 
routes. This finding is discussed further later in the report. The estimated effect size of bilingual teachers from 
postbaccalaureate routes was +0.12 (figure 5 and table 2). To determine the difference in achievement growth between 
students assigned to bilingual teachers from postbaccalaureate routes and those assigned to bilingual teachers certified 
through traditional routes (—0.03), the effect size from traditional route teachers (—0.03) is subtracted from the effect 
size from postbaccalaureate route teachers: 0.12 ——0.03 = 0.15. The effect size from the additional exam route is also — 
0.03 (figure 5 and table 2). 


Figure 5. Bilingual teachers from postbaccalaureate certification routes were 


associated with English learner students’ greatest English proficiency growth in grade 4 
ESL is English as a second language. TELPAS is Texas English Language Proficiency Assessment System. 
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* Significant at p < .05. ** Significant at p < .01. *** Significant at p < .001. 
Note: Data are missing for the postbaccalaureate route for English as a second language (ESL) teachers in the figure due to the 
small sample size for the specific route. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


English as a second language teachers from traditional routes were associated with greater growth in English 
proficiency in grade 4 compared with English as a second language teachers certified through other routes. The 
analyses on English proficiency outcomes in grade 4 suggest that ESL teachers certified through traditional routes were 
associated with greater growth in English proficiency compared to ESL teachers from other routes and compared to 
teachers with no bilingual certification (figure 5 and table 2). Furthermore, ESL teachers from traditional routes also 
were associated with greater growth in English proficiency compared to bilingual teachers from alternative, traditional, 
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and additional exam routes (figure 5). Specifically, two of the three routes to ESL certification for which data were 
available were associated with positive and significant effects on English proficiency growth: traditional (effect size = 
0.09) and alternative (effect size = 0.06) routes. One route to ESL certification was associated with a negative and 
significant effect: additional exam route. The effect size of ESL teachers from traditional routes is 0.09. The effect size 
of ESL teachers from alternative routes is 0.06, (figure 5 and table 2). 


Table 2. The association between bilingual and ESL teachers by certification route 
and student achievement growth varies by grade and subject 


Bilingual certification ESL certification 


Direction and size of effects Direction and size of effects 


Alterna & Postbacca- Alterna- = pos aaa 
Grade and outcome - tive rea PNXololidfoyare}| laureate tive erp Additional 
Traditional Traditional laureate 
exam exam 


Math 0.17*** 0.12*** 0.01 0.06 -0.04 0.02 0.04 -0.05 


Perna Reading 0.03 0.08** -0.01 0.00 -0.05 -0.06 -0.00 a 


enelet 0.01 -0.03* 0.06* 0.09* -0.03* a 
proficienc -0.03* 0.12*** 


y 


Math -0.15** 0.19" ** -0.07 -0.00 -0.19* 0.05 -0.09 
Renn Reading 0.02 -0.03 -0.01 -0.15 -0.01 -0.11 0.01 a 

English 0.04* -0.02 0.00 0.05 -0.00 a 

proficienc -~0.02 0.10 

y 


ESL is English as a second language. 
* Significant at p < .05. ** Significant at p < .01. *** Significant at p < .0O1 a: data are missing due to the small sample size for the specific 
route. Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


The analyses on English proficiency outcomes in grade 5 suggested that bilingual teachers certified through alternative 
routes were associated with the most growth in English proficiency compared to bilingual and ESL teachers from other 
routes and compared to teachers with no bilingual or ESL certification (see figure 6). Specifically, only one of the four 
routes to bilingual certification was associated with a positive and significant effect on English proficiency growth: 
alternative. Students assigned to bilingual teachers from traditional, additional exam, or postbaccalaureate routes did 
not experience significantly different growth in English proficiency compared to students assigned to teachers with no 
bilingual or ESL certification. Having an ESL teacher regardless of route was not associated with English proficiency 
growth in grade 5. The estimated effect size of bilingual teachers from alternative routes on students’ English 
proficiency growth was 0.04 compared to students taught by bilingual teachers certified through any other route, or by 
ESL teachers from any 

route, or by teachers with 0.10 

no bilingual or ESL 
certification (figure 6 and 
table 2). 
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ESL is English as a second language. TELPAS is Texas English Language Proficiency Assessment System. 
* Significant at p < .05. ** Significant at p<.01. *** Significant at p<.001 
Note: The postbaccalaureate route for ESL teachers is not included in the figure because no observations were available. 
Source: Authors’ analyses of data provided by the Houston ISD and the ERC at the University of Texas at Austin. 


Policy Recommendations/Conclusions 


The following results were found: 

* Having a certified bilingual teacher was associated with greater growth in math and reading for English learner 
students in grade 4 and lower growth in math for English learner students in grade 5 compared to having a teacher 
not certified in bilingual or ESL. 

¢ There was no evidence that having a certified bilingual teacher contributed more or less to grade 4 or 5 English 
learner students’ growth in English proficiency. 

* There was no evidence that having a certified English as a second language teacher contributed more or less to 
grade 4 or 5 English learner students’ growth in math, reading, or English proficiency. 

¢ Students assigned to bilingual teachers from the additional exam and postbaccalaureate routes did not experience 
significantly different growth in math compared to students assigned to teachers with no bilingual or ESL 
certification. 

¢ Bilingual teachers certified through alternative routes were associated with greatest growth in grade 4 math 
compared with teachers certified through other routes. 

¢ Bilingual teachers from the additional exam route and ESL teachers from traditional routes were associated with the 
least math achievement growth in grade 5 compared with teachers certified through other routes. 

¢ The analysis was not able to detect an association between ESL teachers certified through alternative, additional 
exam, and postbaccalaureate routes and math achievement growth in grade 5. 

¢ Bilingual teachers from traditional routes were associated with greatest growth in grade 4 reading compared with 
teachers certified through other routes. 

* Teacher certification route was not associated with teachers’ contributions to English learner students’ grade 5 
reading achievement. 

¢ Bilingual teachers from postbaccalaureate routes were associated with greatest growth in grade 4 English 
proficiency compared with teachers certified through other routes. 

* Bilingual teachers from alternative routes were associated with greatest growth in grade 5 English proficiency 
compared with teachers certified through other routes. 


The results suggest that being assigned a certified bilingual reading teacher did not contribute significantly more or less 
to English learner students’ English proficiency compared to teachers who are not certified bilingual or ESL in either 
grade 4 or grade 5. Also, being assigned a certified ESL teacher did not contribute significantly more or less to grade 4 
or grade 5 English learner students’ math or reading achievement growth compared to being assigned a teacher who 
was not certified bilingual or ESL (the reference group). Likewise, ESL-certified teachers did not contribute 
significantly more or less to English learner students’ English proficiency compared to teachers who were not certified 
bilingual or ESL in either grades 4 or 5. 


The results suggest that for grade 4 teachers of English learner students whose home language is Spanish, it may be 

preferable to assign bilingual teachers certified through alternative routes to teach math; bilingual teachers certified 

through traditional routes to teach reading; and bilingual teachers certified through postbaccalaureate routes to work 
with English learner students in need of support with English proficiency. 


When considering allocating teachers to English learner students whose home language is Spanish in grade 5, the 
results suggest that bilingual teachers were associated with lower growth in math, and they were not associated with 
significantly different growth in reading compared to teachers with no bilingual or ESL certification. These results 
would support school leaders allocating scarce bilingual teachers to teach math and reading to grade 4 students but 
allocating bilingual teachers certified through alternative routes to work with grade 5 English learner students in need 
of support with English proficiency. 
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